SOFTWARE ENABLING SCHOOL IMPROVEMENT 
THROUGH ANALYSIS OF STUDENT DATA 



Jeffrey C. Wayman 
Sam Stringfield 
Johns Hopkins University 

Mary Yakimowski 
Baltimore City Public School System 



Report No. 67 



January 2004 



This report was published by the Center for Research on the Education of Students Placed At 
Risk (CRESPAR), a national research and development center supported by a grant (No. R117- 
D40005) from the Institute of Education Sciences (IES, formerly OERI), U.S. Department of 
Education. The content or opinions expressed herein do not necessarily reflect the views of the 
Department of Education or any other agency of the U.S. Government. Reports are available 
from: Publications Department, CRESPAR/Johns Hopkins University; 3003 N. Charles Street, 
Suite 200; Baltimore MD 21218. An on-line version of this report is available at our website: 
www.csos.jhu.edu. 

Copyright 2004, The Johns Hopkins University. All rights reserved. 




The Center 



Every child has the capacity to succeed in school and in life. Yet far too many children 
fail to meet their potential. Many students, especially those from poor and minority fami- 
lies, are placed at risk by school practices that sort some students into high-quality pro- 
grams and other students into low-quality education. CRESPAR believes that schools 
must replace the “sorting paradigm” with a “talent development” model that sets high 
expectations for all students, and ensures that all students receive a rich and demanding 
curriculum with appropriate assistance and support. 

The mission of the Center for Research on the Education of Students Placed At 
Risk (CRESPAR) is to conduct the research, development, evaluation, and dissemination 
needed to transform schooling for students placed at risk. The work of the Center is 
guided by three central themes — ensuring the success of all students at key development 
points, building on students’ personal and cultural assets, and scaling up effective pro- 
grams — and conducted through research and development programs in the areas of early 
and elementary studies; middle and high school studies; school, family, and community 
partnerships; and systemic supports for school reform, as well as a program of institu- 
tional activities. 

CRESPAR is organized as a partnership of Johns Hopkins University and Howard 
University, and is one of twelve national research and development centers supported by 
a grant (R1 17-D40005) from the Institute of Education Sciences (IES, formerly OERI) at 
the U.S. Department of Education. The centers examine a wide range of specific topics in 
education including early childhood development and education, student learning and 
achievement, cultural and linguistic diversity, English language learners, reading and lit- 
eracy, gifted and talented students, improving low achieving schools, innovation in 
school reform, and state and local education policy. The overall objective of these centers 
is to conduct education research that will inform policy makers and practitioners about 
educational practices and outcomes that contribute to successful school performance. 
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Abstract 



The No Child Left Behind legislation has drawn increased attention to student data. Data 
are most useful in educational decision-making when the purpose extends beyond vertical 
accountability and toward school- and classroom-level decision-making that enhances the 
experience and achievement of students. This necessarily involves getting practical data 
analyses into the hands of teachers and administrators. Recent technological advances in 
data warehousing and presentation have resulted in tools that can, in theory, facilitate 
educator use of student data. However, the use of these tools is not yet widespread. The 
resulting condition is ripe for both educational improvement and research. In this report, 
the authors consider issues surrounding the use of student data and data based decision- 
making, describing the state of the field and possible future directions, present reviews of 
a range of commercially available software for analyzing student data, and provide and 
maintain a website that will contain ongoing updates of software reviews. 
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A Near-Term Vision 



Ms. Lockhart, a middle-school teacher, arrives at her desk with 15 minutes to spare be- 
fore her academic day begins. She checks the control panel on her desktop computer. 
Fernando and Jamaal will be absent (their parents have phoned or emailed); Susan will 
be back from four days of family vacation. Both Susan and her mother promise that she ’ll 
make up her schoolwork over the weekend. 

Last night, Ms. Lockhart finished grading her third-period students’ projects. She 
uploaded their grades into her grade book. Projects submitted electronically were up- 
loaded into students ’portfolios. On the way to her office, she left the other projects at the 
front desk to be scanned in by a first-period student worker. 

Scotty ’.s' work had been particularly troubling to her, with odd spelling errors and 
occasional words omitted. This morning, she opens his electronic portfolio and notices 
similar problems in his third- and fifth-grade products, with substantial variance in 
achievement test scores over time. Ms. Lockhart doesn 7 have a lot of time today, so she 
sends an email to the special education diagnostic team, asking that someone look at the 
latest document and at Scotty ’s prior work and test scores. 

She quickly opens today’s lesson set: English grammar for most students, with 
three students retaking a quiz they failed last week and three other students writing extra- 
credit book reports. 

The bell rings, her students arrive, and first period begins. Janie is absent for the 
fourth consecutive day. Ms. Lockhart ’s completion of the electronic roll book will auto- 
matically signal the main office to contact Janie’s family. 

With 15 minutes to go in the period, Ms. Lockhart informs the students that their 
homework assignments are both on the screen at the front of the class and in their email 
files. She checks the three students ’ retaken quizzes, congratulates two (and enters their 
new grades in her electronic grade book), and assigns a different type of review to the 
third student. Ms. Lockhart emails the third student ’s mother, asking her to double-check 
the boy ’s homework tonight, and to email back if there are things the teacher should par- 
ticularly attend to over the next week. 

With two minutes to go in the class, Ms. Lockhart informs students that their end- 
of-term grades will be available in their web folders and on paper in a week. She will be 
asking several students and a few parents to come in for shared problem solving before 
the next semester begins. Most of the class, she says reassuringly, is doing well, and sev- 
eral students have shown remarkable progress this semester. 

A look at her computer desktop reveals a new message: Janie’s family moved 
without giving notice to the school. Ms. Lockhart sighs and writes a brief email to the re- 
ceiving teacher, attaching a copy of Janie ’s permanent record to the email. She tells the 
class that they won 7 be seeing Janie any more, but that she is well and at another school 
in the district. She offers to forward any '‘goodbye” email messages that anyone would 
care to send. 
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Ms. Lockhart uses a few extra minutes at the end of her day to explore some data 
disaggregation. She finds that her students on free lunch have been improving on both 
the state and local assessments this year. She is happy about that, since narrowing the 
achievement gap between her advantaged and less-advantaged students was one of her 
goals at the beginning of the year. 

The bell rings and 28 preteens bound out of their seats and into the hall, talking 
and laughing and moping and picking on each other, just as their grandparents did 50 
years earlier. In several senses, nothing is different about this school scenario. Parents 
still pull children out of school to visit grandparen ts or doctors or to go to Disneyland. 
Teachers still don ’t have enough time to meet with special education specialists, or to go 
to the office to look through a child’s permanent record. 

Yet a great deal is different. Ms. Lockhart keeps her attendance and grade book 
electronically. At a moment ’s notice, she can not only see students ’ products from her 
classes, but also view products from years gone by. She can forward a special education 
initial referral to the proper team with a few mouse clicks. Parents can leave messages 
for teachers, and by 9 a.m. every day can double-check that their children actually 
showed up for their classes. The technology that assists Ms. Lockhart is doing several of 
the things technology does best — efficiently storing and retrieving data, and moving rele- 
vant information to the people who can most professionally act upon it. Ms. Lockhart, 
who was not significantly better prepared to become a teacher than her predecessors of 
10 years earlier, is nonetheless a significantly more efficient, more effective educational 
professional. 
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Introduction 



N.L. Gage, editor of the first Handbook of Research on Teaching (1963), regularly states 
that he has been told throughout his long career: “Computers are going to totally trans- 
form education — it’s just around the corner.” He reports that a 50-year corner is some- 
thing at which to gaze in wonder (Gage, personal communication). Computers offer a 
wide array of capabilities and uses that are directly applicable to education, so why have 
computers not yet transformed education? Why has their implementation in our class- 
rooms and schools been slow? Why is our vision for Ms. Lockhart a vision, and not a re- 
ality? 

We termed our vision for Ms. Lockhart a “near-term” vision because we believe 
events are aligning that suggest that educators may soon begin turning Gage’s 50-year 
comer. Today there is a clearly heightened emphasis on school use of data. The account- 
ability requirements of the federal No Child Left Behind (NCLB) legislation have only 
increased this trend. 

Software companies are increasingly interested in the education market and are 
continually introducing improved technology into the market. Perhaps most importantly, 
advances in computer hardware and software have made practical solutions to data use 
dramatically more cost-efficient. We all readily witness the remarkable increase in raw 
computing power and greatly reduced costs that have marked computer development 
over the years. 

We believe that the potential for an impactful use of computers lies in putting the 
great quantities of student data already owned by schools into practical service. Schools, 
districts, and states have collected large amounts of student data for years, but this data is 
typically not put to classroom use. It is not unfair to suggest that today’s schools are data- 
rich but information-poor. 

Our belief is that teachers would make valuable use of data to inform their class- 
room practice if such data were quickly and easily available to them in forms that fit their 
needs and answered their questions. Software packages to facilitate such data use are cur- 
rently in the market, and these packages can be implemented by wealthy and impover- 
ished districts alike. In this report, we describe the software possibilities that exist in this 
area and explore what is potentially available for the foreseeable future. 



Research on School Data Use 

Research on school improvement and school effectiveness has shown data use to be cen- 
tral to the school improvement process (Chrispeels, 1992; Earl & Katz, 2002), and there 
are many case studies available describing the variety of ways in which data has sup- 
ported educational decisions (e.g., American Association of School Administrators, 2002; 
Feldman & Tung, 2001; Lachat, 2002; Pardini, 2000; Protheroe, 2001). Recent policies at 
the federal, state, and local levels have served to bring data use to the fore. As Earl and 
Katz (2002) note, data use is now not a choice for school leaders, but a must. 
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Data can be used to inform solutions to a wide variety of educational challenges. 
Streifer (2002) listed exploring group differences, exploring growth over time, program 
evaluation, and identifying root causes of educational problems as being among the many 
ways data can be used. A study by Chrispeels, Brown, and Castillo (2000) showed data 
use to be a strong predictor of the efficacy of school improvement teams. Data use not 
only increased efficacy directly, but served as a mediator for the positive effect of other 
factors. Kennedy (2003) included use of data as a central component of his model for 
raising achievement test scores. 

Data also can have a positive effect on the people involved in the educational 
process. Feldman and Tung (2001) observed that schools involved in data use often 
evolved toward a more professional culture. Educators in their study became more col- 
laborative during the data/decision process, and school business consequently became 
less “privatized.” Earl and Katz (2002) note that school leaders involved in data use of- 
ten develop a mindset of being in charge of their own destiny, increasingly able to find 
and use information to inform their school’s improvement. Armstrong and Anthes (2001) 
found that data use was helpful in raising teacher expectations of at-risk students, noting 
positive changes in teacher attitudes regarding the potential success of previously low- 
performing students. 

Although data use provides many positives, the process of increasing data use in 
schools is not necessarily an easy one. One obstacle, in particular, involves technology. 
Although computers supporting knowledge management have been in widespread use in 
areas such as business, Thom (2001) states that schools present difficult technical prob- 
lems because of the variety of data needs and uses in school organizations. School data is 
often in disparate forms and locations, making it difficult to organize into an efficient da- 
tabase. To underscore this point, Thorn described a case study where a district was ready 
to implement data based decision-making, but technological barriers hindered the proc- 
ess. Recent technological advances are helping schools overcome these technological 
barriers. Stringfield, Wayman, and Yakimowski (2003) forecast that schools soon will 
have a variety of affordable, efficient computer tools to aid in the data process. 



Information Management 

Schools and districts have a new opportunity to provide professional educators, students, 
and their parents access to large amounts of student information. Today, schools can en- 
able key decision-makers (e.g., teachers, principals) with data and information to facili- 
tate more informed decision-making and improve school performance. 

This concept is not new. Known broadly by many terms, depending on the field 
(e.g., “information management,” “knowledge management,” “data based decision- 
making”), this concept is indispensable in non-educational settings such as business and 
industry. In several areas of our society, ranging from retail store chains to airlines, 
highly sophisticated tools for delivering infonnation to those in position to use that in- 
formation are already part of standard business practice. 
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The last major impediment to widespread implementation of data use for educa- 
tional decisions exists at the software level, and we believe this hurdle can be cleared 
soon. Technological advances in the areas of data warehousing and delivery have been 
accomplished in many other areas and software companies are beginning to focus their 
attention toward the development and marketing of efficient, easy-to-use products for 
school use. Simultaneously, educators are becoming more aware of the application of 
data use for decision-making that benefits students beyond mere accountability measures 
such as the NCLB act. Thus, we believe the pieces are in place for information manage- 
ment to provide significant advances in educational effectiveness. 

Information management (IM) has several useful applications for school person- 
nel who want to use student data to improve instruction: 

• By storing and retrieving varied performance data on individual students, IM technol- 
ogy can substantially increase the information available to professional educators. 

• By easily and quickly generating standard reports on classes and schools, IM technol- 
ogy can provide useful information within and across classes and schools in fonnats 
that educators at all levels can quickly share and use to seek “best practice.” 

• By facilitating the generation of unique reports fitted to the questions of an individual 
educator or group, IM technology can aid in knowledge-based decisions tailored to a 
specific context (e.g., schools, teachers, localities). 

• By making daily and annual infonnation available to parents, mediums such as re- 
ports and web-based data presentations can increase parental involvement in every- 
thing from nightly homework to long-term educational planning. 

• By making a broad range of multilevel aggregated and disaggregated data available to 
principals and central administrators, IM technology can provide increased opportuni- 
ties to examine and understand factors affecting their schools’ progress or lack 
thereof. 

• By making available to teachers infonnation previously obtained only through poring 
over hard copies of student records, IM technology can enable increased familiarity 
with students and help inform classroom practice. 

• By making a broad range of student data easily available to teachers, IM technology 
can help teachers become classroom researchers. 

• By storing years of students’ actual work in such diverse areas as writing, mathemat- 
ics, and art, IM technology has the potential to offer a comprehensive portfolio of 
student work and progress, and have these data available in real-time to professionals, 
parents, and the students themselves. 

That a comprehensive solution for widespread data use in education is not yet in 
the market presents a unique opportunity for school stakeholders and educational re- 
searchers to help shape the future of this technology. Educational settings present a dif- 
ferent set of challenges than those in, say, business, so software companies will need to 
research the field to provide tools that are appropriate for widespread use in education. 
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For IM to be a truly beneficial enhancement for educational decision-making, it is impor- 
tant that the market for educational IM technology be driven by the needs of education, 
not vice-versa. Educators can help guide IM innovation by becoming discerning consum- 
ers, making educated choices in the software they buy and helping software developers 
learn exactly what schools need to provide the best education for our students. In the next 
section, we will discuss a series of issues that we believe anyone interested in the use of 
IM solutions might wish to consider. 



Important Issues In Software Implementation 

Implementation of a student data analysis and presentation system can present many un- 
foreseen challenges, so in this section we examine some topics surrounding system im- 
plementation. 1 

Issues discussed in this section include a thorough assessment of basic data needs, 
time to implementation, cost, choosing a vendor, and the Schools Interoperability 
Framework (SIF). Also addressed is the fact that many schools will be faced with the 
choice of whether to “build” (implement the system using district staff), “buy” (purchase 
a commercially available product and services), or develop a hybrid of the two. 

Assessment of Data Needs 

The first step toward implementing a student data analysis system should be to thor- 
oughly assess the data, needs, and resources available to a school or district. This in- 
cludes taking an exhaustive inventory of data sources, surveying analysis needs and 
goals, and choosing whether to implement the data system using local staff or contracted 
help. 

Data inventory and preparation. Data considerations start with an inventory of 
the data currently stored — a cataloging of what data exists and where it is located. It is 
not uncommon for schools to have data in different locations, ranging from a main dis- 
trict data store, to an Excel spreadsheet on a counselor’s computer, to loose papers in a 
teacher’s files, so it is important to identify every source and location of data. This explo- 
ration necessarily interacts with the running inventory of what data will ultimately be 
made available for analysis. 

Data “cleaning.” No analysis system provides value if the underlying data are 
inaccurate, and school data invariably contain problems, often large and vexing ones. 
Therefore, schools should assess the quality of the extant data, and estimate resources 
required for “cleaning” the data (e.g., correcting errors, omissions, and redundancies) so 
that the data can be integrated. Both school districts and vendors report that one immedi- 
ate benefit from implementing a data analysis system is the opportunity to clean and im- 
prove existing data sets. This often includes unanticipated benefits that emerge when 



1 A previous examination of some of these topics can be found in a publication from the National Center 
for Education Statistics (Clements, 2000). 
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people at different levels become engaged in the cleaning and data set integration proc- 
ess. 

Software needs. Currently available software presents a wide range of specializa- 
tion. Some products offer strengths in data efficiency, others in data presentation or 
graphics, but no current product offers strengths in all areas. Therefore, school systems 
must carefully evaluate their data and analytic needs to identify appropriate software that 
best fits their local needs. 

School data analysis systems should provide accurate and efficient storage and re- 
trieval along with useful and intuitive presentation. Efficient storage is often accom- 
plished through a “data warehouse,” a system that integrates disconnected sets of data 
often found in schools into one large, seamless database. It is important that warehousing 
be accomplished in such a way that data retrieval is efficient. Systems that provide fan- 
tastic presentation of data with slow retrieval or cumbersome data management are not 
useful. The same is true of systems that provide excellent data management and access 
but poor data presentation. Additionally, evaluation should be done of the capacity of a 
data system to deliver longitudinal infonnation as the database grows in the future to in- 
clude many years of student data. 

Schools should also take care to assess the types of data, presentation, and analy- 
ses that will be most helpful in educational decisions in their context. There is a paradox 
here, however: even well-trained professionals seldom know exactly what they will find 
most useful, and what they will need, until they are well into their work. Therefore, we 
suggest that schools cast a wide net in this assessment, involving input from many types 
of school personnel. Development of software for analyzing student data is in its infancy, 
so school systems’ assessments of software needs will undoubtedly evolve, as the capa- 
bilities of various options become more widely known. We thus envision the software 
needs assessment as evolutionary in its own right, and we recommend that schools assess 
their data and software needs with an eye toward future expansion and flexibility. Failure 
to plan for this inevitable evolution may well result in the system offering undue con- 
straints on data use. 

What does this mean for the district or school that wishes to start in the near fu- 
ture? In terms of currently available software, there is no “best” program for analyzing 
student data because the strengths and features of current software packages vary greatly. 
Many programs presently in use, however, provide valuable student data analyses, and it 
is possible for schools to choose adequate programs today while also planning to take ad- 
vantage of future technological advances. 

Outside help. Some schools have relatively clean data or have the necessary per- 
sonnel to quickly organize and substantially improve the quality of their data sets. Other 
schools/systems have data that would require a great deal of attention, and the amount of 
work required for data preparation and cleaning may seem intimidating. The experience 
of practitioners across the country indicates that the majority of schools and school sys- 
tems will require substantial effort to successfully launch a practically useful, accurate 
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data system. Many districts and software companies have been through this process, and 
it is possible to leam from the hard-won experience of others. 

Third party assistance is also available in several areas. Several commercial ware- 
housing and analysis companies offer help with the initial data inventory, helping schools 
identify potential data sources and aiding in merging data from dissimilar formats. Addi- 
tionally, though the cost of adequate computing power and data storage capacity contin- 
ues to drop dramatically, these costs may yet be seen as expensive for some schools or 
districts. Many commercial warehousing vendors offer affordable options for housing 
school data, promising fast access across the Internet. 

In assessing data needs, schools and districts ultimately must make a decision as 
to which efforts are handled best within the organization and which must be outsourced. 
Our belief is that unless a district is certain that the expertise to deal with data problems 
quickly and efficiently exists in-house, the experience that an outside organization brings 
to the process is well worth the cost, especially when time and accuracy are considered. 

Time to Implementation 

Rapid, successful implementation is important for the long-term development of a data 
based decision-making climate. Experiencing early success is a correlate of long-term 
implementation success in a range of change efforts (Fullan & Miles, 1992), so speedy 
realization of at least some aspects of a useful information management system will sus- 
tain interest and improve education sooner rather than later. Many of those interviewed 
for this chapter stressed the importance of getting data “up and running” quickly, even if 
less than a full orchestra of data or reports is initially available. 

The amount of time required to achieve implementation is a major consideration 
in choosing whether to build or buy a data system. Developed locally, implementation of 
a data warehouse with reporting and analysis capabilities often takes years, while most 
commercial vendors promise an established (i.e., running and useful) product in a matter 
of months. Competent commercial vendors can usually bring practical experience and 
specialized staffing, and thus can often get a system established and functional much 
faster than school personnel can when building it themselves. This is not a criticism of 
local school technology personnel, it merely reflects the efficiency advantages when an 
organization or business specializes in a particular product and set of processes. 

The best route to rapid implementation is one of the important choices a school 
system faces in launching such a system, and it is beneficial to know that there are multi- 
ple options available to achieve this objective. 




Cost 

Dollar cost structures of currently available software can be complicated and are thus dif- 
ficult to accurately establish in this report. However, light can be cast on a few areas con- 
sumers should consider when evaluating costs. 

For commercially available systems, the final dollar cost depends on many vari- 
ables. Depending on the size of a district and the number of features chosen, the cost for a 
school to implement a system could range from as low as two to more than ten dollars per 
student per year, sometimes with a higher cost in the first year of implementation. Per- 
student costs are generally lower for larger numbers of students. Some vendors have 
noted arrangements where smaller districts share a system to take advantage of cost re- 
ductions. 

Opting for external assistance with data collection and cleaning typically carries 
an added cost, as does opting for outside data storage. Some companies charge for con- 
sulting time spent with representatives, others build an assumed amount of consulting 
time into per-student pricing. In outlining a proposal, vendors may package various op- 
tions together for one cost, or many of these features and others (e.g., access to state stan- 
dards infonnation) may be included in the basic system. Whether bundled together, cho- 
sen separately, or included in the basic system, a hardly surprising general rule is that 
more features translate to higher costs. 

Costs for implementing a locally developed system are typically more difficult to 
evaluate. Such costs include deployment of hardware and other materials, along with 
salaries and benefits for local employees used in the project. Not only must local talent be 
paid (a cash cost), but the time they spend developing, debugging, and implementing the 
product is time that could be spent on other projects (an ongoing opportunity cost). These 
other projects will then either be ignored or pursued with other staff, both at some real 
but typically not quantified cost. Data cleaning presents other cost issues. In our inter- 
views with local educators and national product developers, we consistently heard that 
school data is almost always “messier” than school personnel anticipate. Schools plan- 
ning local implementation should fully assess the quality of their data and the ability of 
the personnel structure to handle this project in the short and long terms. Additionally, 
there is no guarantee that the locally developed product will initially work in the ways 
intended, so development costs past initial implementation generally serve to further 
complicate the picture. Consequently, a clear understanding of the school organization 
and issues surrounding implementation of such a system is vital to accurately evaluate the 
cost of local development. 

Time must be evaluated as a cost in an additional way. Schools are about the work 
of education, and in education, lost time carries costs. When contrasting a “less expen- 
sive” locally-built data tool taking two years to develop and implement with a “higher 
cost” commercially-built tool taking three months to implement, one well might conclude 
that the locally-built product is “cheaper.” But does the lost opportunity to better diag- 
nose and educate children justify the dollar savings? Each day and year that data goes 
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unused is a lost opportunity to better diagnose and educate children. In evaluating costs, 
school entities should include opportunity costs. 

Choosing a Vendor 

Should a district wish to purchase a commercially available student data and analysis sys- 
tem, there are many products from which to choose. Software companies focus on differ- 
ent strengths, and no company currently offers a software package that is strong in every 
area. Today there is no “best product,” so choosing a vendor entails finding the best fit 
for the local district’s or school’s needs. 

In choosing a vendor, school systems first should evaluate the range of services 
they wish to purchase. Some vendors offer services that schools may find useful, beyond 
delivery of data. For instance, a school in need of help with educational improvement, or 
a school in need of professional development for data use, might do well to contract with 
companies that provide these added services. 

Schools and districts would be well advised to contact other schools using the 
software, query them thoroughly, and visit as many as possible to get practical feedback 
on the types of products and levels of service each vendor offers. Schools’ own negotia- 
tion experience with the vendor may be significant. Does the company return calls in a 
timely manner? Is the company forthright and responsive to infonnation requests? Any 
prospective purchaser would be well advised to establish multiple relationships at the 
prospective company (e.g., both a salesperson and a relatively senior executive or techni- 
cal specialist). This not only helps school personnel learn about the company, but will 
provide multiple routes to resolution should problems arise. 

Finally, there are issues related to the long-tenn viability of various companies. In 
a start-up industry, no one can be sure which companies will stand the test of time. One 
possible guide is the establishment of industry standards, currently being explored 
through the Schools Interoperability Framework, discussed in the next section. 

Schools Interoperability Framework 

The Schools Interoperability Framework (SIF) is a collaboration of school data stake- 
holders that set data exchange standards to enable software packages to communicate 
without further software intervention. The SIF website (www.sifinfo.org) identifies SIF 
as “an industry initiative to develop an open specification for ensuring that K-12 instruc- 
tional and administrative software applications work together more effectively.” 

SIF came into existence because stakeholders in the technology industry recog- 
nized that school options for software were growing rapidly, and that if no standards were 
set for inter-software communication, schools could potentially be handicapped in their 
options for improving and upgrading their data capabilities over time. At the time SIF 
was formed, school personnel were experiencing such problems as redundant data entry, 
loss of data, and an increasing amount of time spent on data entry and management. 
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Without industry standards to ensure communication, it is feared that school systems’ 
ability to send and receive data efficiently (e.g., to and from state departments) could be 
greatly hampered. Consequently, SIF members have engaged in ongoing work to estab- 
lish industry standards in defining fonnats for shared data, naming conventions, and rules 
of interaction among software applications (www.sifinfo.org). The current list of SIF 
members includes educational technology companies, school districts, federal and state 
government offices, and research organizations. 

The SIF initiative is an ongoing project. Accomplishments include the develop- 
ment of a Zone Integration Server that enables software programs to communicate with 
one another, and the establishment of definitions and objects for commonly used student 
data (e.g., name, address, gender), and most recently, the launch of the SIF Compliance 
program in May 2003. The SIF Compliance program allows for applications to be tested 
by a third party validation system to ensure both schools and vendors that applications 
developed using the SIF standard will work together. More details, including the list of 
SIF-compliant applications, can be found at www.opengroup.org/sif/cert/cert_prodlist.tpl. 
Ongoing efforts include establishing forms and definitions for “learning data” such as 
grades and assessment information. SIF is also working with a number of states and the 
U.S. Department of Education to enable SIF to support district-to-state reporting in sup- 
port of No Child Left Behind (NCLB). 

Vendors hold a wide range of attitudes toward SIF compliance. Many are solidly 
supportive of the process, and some have already completed SIF certification for their 
products. Others express caution due to the cost of participation. Other personnel ex- 
press apprehension as to the ultimate value of SIF standards. Some companies report 
waiting to pursue SIF standards until there is more demand from their clients. In addition, 
some representatives of companies not listed as SIF-certified maintain that although their 
company has not yet participated in SIF certification, their products are nonetheless com- 
pliant with SIF standards. 

The impact SIF holds for schools in the process of buying software is the assur- 
ance that SIF-compliant applications will be able to communicate with each other. No 
school wishes to be in the position of depending on a software package that later proves 
to be incompatible with other software. Consequently, if SIF definitions become the in- 
dustry standard, buying an application from a company that does not place emphasis on 
SIF compliance could place a school or system in the awkward position of having to 
change technologies. 

In general, school personnel should take care to protect their software investment 
against future isolation by ensuring that this product can communicate with other prod- 
ucts. SIF represents a promising initiative, and SIF standards represent a reasonable start- 
ing point in evaluating software interaction. In evaluating software, school personnel 



2 As of this writing, initial SIF certification costs are $35,000 for a non-SIF member 
(www.opengroup.org/sif/cert/docs/fees.htm). 
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should thoroughly question software personnel about knowledge and involvement with 

3 

SIF, along with plans of how their software will grow to gain compliance. 



What Should Good Software 
for Student Data Analysis Look Like? 

The most efficient software for student data analysis would be an intuitive, user-friendly 
package that provides comprehensive, error-free data to users at every level. In this sec- 
tion, we describe software features that should be present in such a package. Table 1 pro- 
vides a synopsis of our narrative. 

Table 1. Important Features of a Student Data Analysis System 



User Friendliness 

Software is intuitive and easy to use. 

Software requires little training. 

Presentation is familiar to user. 

Access speed is fast and efficient. 

User Features 

Comprehensive query tools available for every level of user. 

Flexible drill-down capability from any form of data aggregation. 

Data can be accessed from anywhere. 

Information Access 

Multiple ways to access information. 

Varied methods of representing information (e.g., tables, graphs). 
Wide range of data available for analysis. 

Interface provides immediate access to relevant information. 
Pre-formatted reports are clear, varied, relevant, and comprehensive. 
Longitudinal presentation of data available at every user level. 

Creating and Sustaining Quality Data 

Provides capacity to enable clean data. 

Company accepts responsibility to facilitate data process with schools. 
System allows for expansion past initial implementation. 

System provides proper security for data transmission. 

Integration of different areas of information is seamless to the user. 
Software accepts many common data formats. 

Additional Features 

Online student work samples available. 

Software exports into common programs. 

Users can access electronic discussion groups. 

Easy access to learning standards information. 

Software offers capacity to link individual teacher data to student data. 



3 School personnel may access a full set of questions regarding SIF compliance, also suitable for use in an 
RFP, on the SIF website at www.sifmfo.org/getrfp.html. 
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User Friendliness 

To provide maximum value, software for student data analysis must be easily accessible 
by all levels of educators. We believe that educators will have little tolerance for pro- 
grams that are limiting, frustrating, or esoteric, so the best product will be a “user 
friendly” one that intuitively provides a set of data analyses to the user. A program with 
an intuitive, easy-to-use interface will serve to promote data access and increase educator 
desire to pursue the substantial range of information available in student data. 

Given the time demands educators face, and the fact that many are not computer 
experts, it is logical to assume that teachers and other school professionals will make 
more and better use of analytic software if it is presented in a familiar form that requires 
little or no training. For example, the growing use of the Internet has made many users 
comfortable with web-form elements (e.g., links, check boxes, and pull-down menus 
common to sites such as Amazon.com). User-friendly features such as these entail nomi- 
nal training time for users and serve to minimize frustration. 

School data warehouses are large and will be accessed by many users, even in 
small schools or districts. These conditions raise the issue of the speed at which the sys- 
tem can access data and return answers. It is important that computer response time is 
rapid, enabling deeper data querying. Slow response time is likely to cause user frustra- 
tion and almost certainly reduce data use. 

User Features 

To encourage and facilitate data use, it is important that user features are flexible, allow- 
ing for data access in a variety of forms. 

Users should be able to explore the data and answer their own questions, as op- 
posed to relying entirely upon distributed information. Customized, ad-hoc data requests 
are facilitated through the use of a “query tool.” Although query tools often exist for ad- 
vanced users, or “power users,” it is equally important that software offer query tools for 
less sophisticated users. Query tools should be simple to use and require little training, 
incorporating user-friendly features such as those described previously. Query tools 
should also be unrestrictive, allowing access to a wide range of data and the ability to 
provide simultaneous analysis of many variables. 

Broad “drill-down” capability (e.g., the ability to query a school level finding to 
efficiently examine a subset of data at a grade, classroom, or student level) is a desirable 
feature that provides maximum user ease and flexibility. Drill-down functions should be 
available from any point in the program; users should be able to click on graphs, tables, 
or any form of disaggregation to gain more granular infonnation, reaching as far down as 
the student level. 

Educators also must be able to access such software by Internet from home or 
anywhere else they choose to work. Teachers and other education professionals often 
work outside school walls, and it is important to provide data access whenever or wher- 
ever they choose to work. 
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Information Access 

Access to student infonnation should be varied and comprehensive to allow for thorough 
inquiry. Users are limited when the information available does not answer all of their 
questions; further, users will grow in their analytic abilities only if provided with a wide 
range of information fully applicable to their situation. Therefore, the user should be pro- 
vided with wide-ranging means of accessing information, across a range of complexities 
that include every level of user, and with many forms to represent this information (e.g., a 
variety of graphs and tables). Items available for any analysis should never leave the user 
wanting more. 

When the user logs on, the interface should provide quick access to information 
through a variety of methods. Such access may involve quick snapshots of information, 
the aforementioned user query tool, and pre-fonnatted reports on important topics. In- 
formation available in pre-fonnatted reports should be concise, relevant, and comprehen- 
sive. Such reports necessarily must be constructed through a process of thorough consul- 
tation with a diversity of school personnel. 

Students’ education takes place over a number of years, so the range of data 
available must also allow for longitudinal presentation over all periods of available data. 
Longitudinal access enables the user to examine trends of schools, classrooms, and stu- 
dents over time. Presentation and analysis of longitudinal data should be limited only by 
the available data, not by the software itself. 

Creating and Sustaining Quality Data 

The presentation and quality of data are inextricably connected. Regardless of the ease 
and breadth of data access, any analysis is worthwhile only to the extent that the underly- 
ing data are worthwhile. Therefore, it is important that systems for analyzing student data 
are built to provide error-free, secure data with the flexibility to accommodate future 
needs. 

Many school systems will require assistance in building data sets that lead to edu- 
cational improvement. Any company producing such a program should be prepared to 
assist schools and districts with the problems inherent in school data, whether this be 
through the company itself or a partner. Such assistance should include assistance with 
identification of proper data elements to include in the database, data collection guidance, 
and help with correcting data errors and inconsistencies. 

Since school data needs will change over time, it is important that the data ware- 
house and presentation program allow for expansion beyond the initial implementation. 
Any implementation of a data system necessarily imposes boundaries on the extent of 
data use. If a system cannot change as school data needs change, the system will become 
a constraint upon, rather than a facilitator of, educational improvement. 

Nearly all systems allow access from a distance, but such access carries inherent 
security risks that should be of paramount importance. It is necessary that programs offer- 
ing such access provide airtight security for transmitting student data. School personnel 
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and software vendors are aware of these issues and are constantly improving policies for 
protecting privacy. 

Data warehouses by definition integrate many disparate forms of infonnation 
(e.g., attendance, demographic infonnation, test scores). Integration of these areas should 
be seamless to the user; the more transparent this integration is, the easier navigation will 
be. 

The capacity to use many common formats (e.g., ASCII, XML, Excel) to import 
and export data is also important. School personnel will find restrictive formats limiting 
when data is to be transmitted outside the system, as when preparing NCLB data for state 
agencies. Restrictive formats will also be cumbersome when data is to be imported into 
the warehouse. XML in particular is becoming a popular format for data exchange, and 
many developers have adopted XML as a standard. 

Additional Features 

Other desirable features include the following: 

• Online student work samples: school personnel should be able to access not only 
numeric data on a student, but also see annual samples of each student’s “authentic” 
work online. 

• The ability to export graphs or other results into common programs such as Microsoft 
Word or Adobe Acrobat. 

• Electronic discussion groups or “message boards” where users from any location can 
discuss issues relevant to the software. Yearly or bi-yearly user group meetings are 
also desirable. 

• Easy access to the learning standards that drive local or state assessments. 

• The capacity to link a teacher’s own student data to data accessed by the system. For 
example, a teacher might wish to correlate in-class grades with, say, assessment re- 
sults from a state test. 

In the next section, we offer reviews of commercially available software for ana- 
lyzing extant student data. The authors do not endorse any one software option, and be- 
lieve that each has one or more features that might make it attractive to a district with 
needs well matched to the specific software strengths. 



Software Reviews 

Sampling Methods 

A major goal of the authors in preparing this report is to provide a summary of the soft- 
ware options schools have for analyzing and disaggregating extant student data. Software 
was included if it enabled many different levels of school personnel to analyze existing 
student data for achievement purposes (i.e., any data that is collected and stored by the 
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school, such as achievement tests, demographic data, and portfolios). Software that al- 
lowed only district personnel to view student information was not included, nor was 
software that accessed only school management issues (e.g., attendance, absences) or 
software that required ongoing teacher data entry (e.g., grade books, palm pilots, data 
scanning). 

Commercially available software was identified through various means, including 
literature searches, web searches, and personal conversations with individuals familiar 
with school software. Because of the dearth of summaries of school software, much of 
our data were collected by directly contacting people in the industry and slowly building 
a network of information. This “snowball” strategy was time-consuming, but we believe 
that it has produced a reasonably comprehensive summary of commercially available 
software that is usable and useful in warehousing and analyzing student data . 4 

In evaluating the software, we contacted companies and were put in touch with 
representatives, usually sales or management personnel, who were knowledgeable about 
the available software packages. During these interviews and subsequent software dem- 
onstrations, the authors gathered infonnation using a semi-structured interview protocol. 
Companies were included in this review only if our contacts were able to provide an in- 
teractive demonstration of the software. Demonstrations were conducted over the Internet 
during a phone conversation with company representatives so that our lack of training in 
the use of the specific software was not a limitation. In several cases, a demonstration 
website was also made available to us so we could explore the software on our own for 
purposes of more detailed product examination. 

In contrast, our process of identification of locally developed software was neces- 
sarily not comprehensive, as a large number of local initiatives exist. Instead, we con- 
tacted school districts in Houston, Texas, Cleveland, Ohio, and Broward County, Florida. 
These districts were chosen because they have received substantial publicity regarding 
their locally developed systems. We were unable to get Internet demonstrations of these 
programs, so we will not present reviews of their perfonnance. Rather, we will discuss 
these initiatives in terms of interviews conducted with school personnel. 

Software Not Included 

We sought to examine software that enabled teachers and other school personnel to ana- 
lyze existing student data to improve teaching and achievement. Consequently, there is a 
wide variety of quality school data software that we did not review, such as software 
packages that serve only as student information systems, software used for administrative 
student management, software developed to generate assessment data, or software that 
requires data entry by teachers. Examples of such software packages include Power- 
School from Apple, a student data management system that also facilitates parental 

4 Although employing a thorough process, it is possible that the team has missed some vendors currently on 
the market and that others have entered this rapidly changing market since this report was written. The au- 
thors would be most appreciative of any information on additional products that are, or come to be, on the 
market. 
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communication, as well as Renaissance Learning’s tools for analyzing and creating cur- 
riculum-based assessment data. 

There are a small number of software companies with products we were unable to 
review for this report, but that provide analysis of student data. We note these companies 
in Appendix A in order to provide as thorough a list of software as possible. It is our hope 
that subsequent updates of our work will include these products. 

Limitations 

The companies themselves provided the data for these reviews, and demonstrations 
worked from company-controlled data sets. There are many facets of software that are 
best evaluated through using the software in a practical local setting (e.g. response time, 
database size, querying efficiency), but we were not able to undertake a data collection of 
that magnitude or test all products against a single school system’s data base. 

Overview of Reviews 

Before presenting the reviews, we make some general comments about the programs as a 
group. 

The aims and features of software for analyzing student data are varied, and each 
company has chosen one or two aspects of student data on which to concentrate their 
programming efforts. Companies may choose to focus on providing reports, easy data 
access, or assessment data, to name a few. No one piece of software accomplishes every- 
thing, and some features are more common than others. For instance, most companies 
offer pre-fonnatted reports of student data that can be generated with a mouse click. 
Fewer companies offer stored queries, where the user can perform a customized query of 
the data, then save that query for use by themselves or another user. Only two companies 
reviewed here offer storage and retrieval of student work samples, or electronically ac- 
cessible portfolios of student work. 

These programs do share several features, features that a school should expect in 
buying software for student data management. All of the programs reviewed are web- 
based, and thus offer user access from any Internet connection . 5 All programs offer at 
least descriptive analyses (e.g., means of different groups and sub-groups), and all offer 
the capacity to produce reports based on the disaggregations mandated by the NCLB leg- 
islation. All offer some form of ongoing technical support, and all companies are at least 
aware of the evolving Schools Interoperability Framework (SIF). 

Beyond these commonalities, schools and school systems must choose software 
based on specific software strengths. We anticipate that as the field moves toward scale, 
schools will be able to choose from more comprehensive programs. At present, however, 
it is important that schools carefully evaluate present and future software needs, re- 
sources, and available software options, choosing software matched to local needs. 

5 Note, however, that many implementations of QSP utilize the desktop version, which is PC -based. 
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Table 2. Characteristics of Commercially Available Software 



Software 

and 

Company 


Account 

from 

SchoolNet 


Data Miner 
from 

Chancery 


Data Point 
from 
NSSE 


Ease-e 

from 

TetraData 


EDsmart 


eScholar 


QSP 

from 

CRESST 


Company 

Focus 


Educational 

Technology 


Student 

Information 

Systems 


Educational 

Research 


Educational 

Technology 


Educational 

Research 


Data 

Warehousing 


Educational 

Research 


Version 


4.0 


4.1 


n/a 


4.5 


3.2 


5.0 


4.3 


Pre-formatted reports 


X 


X 




X 


X 


X 


X 


Query tool for less- 
advanced users 


X 




X 


X 


X 




X 


Stored queries 




X 


X 


X 


X 


X 


X 


Online student work 
samples 






X 








X 


User discussion boards 
or user meetings 


X 






X 


X 


X 


X 


Accepts data fonnats in 
addition to ASCII 


X 


X 




X 


X 


X 




Variable set customized 
to fit school needs 




X 


X 


X 


X 




X 


Company will house data 


X 




X 


X 


X 


X 




School may house data 


X 


X 




X 


X 


X 


X 


Company helps collect 
data 


X 


X 


X 


X 


X 


X 




Reports SIF compliance 


X 


X 




X 


X 


X 




SIF-certified 


X 


X 




X 




X 




Number of districts in 
use 


40 


83 


15 


464 


27 


750 


100 



In the following narrative, programs are reviewed in alphabetical order. Each review 
starts with an overview of the company and the focus of their software. Discussion then shifts to 
two sub-sections on user functionality: the availability of pre-formatted reports (standard reports 
that are available for review with one click), and query tools (the interface in which the user is 
able to access the data in ad-hoc fashion). The fourth sub-section addresses the software’s 
compatibility with SIF, followed by a sub-section discussing features that were not included 
under the previous headings. Table 2 provides a side-by-side view of some different features 
available in the programs reviewed for this report, and Appendix B provides “screen shots,” or 
examples of the look and feel of reviewed software. Additionally, since software features and 
functions change rapidly, we are maintaining a website that contains updates of product reviews: 
www.csos.ihu.edu/systemics/datause.htm . The Software Reviews section concludes with a 
segment discussing locally developed warehousing/reporting software. 
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Table 2 (cont’d). Characteristics of Commercially Available Software 



Software 

and 

Company 


Sagebrush 
Analytics, 
powered by 
Swift- 
Knowledge 


SAMS 

from 

Executive 

Intelligence 


Scholar Suite 
from 

SCHOLARinc. 


Socrates 
Data System 
from 
CRM 


STARS 

from 

SchoolCity 


Virtual 
EDucation 
from Edmin 


Company 

Focus 


Data Analysis 
& Reporting 


Educational 

Technology 


Management 

of 

Assessment 

Data 


Educational 

Research 


Educational 

Technology 


Learning 

Management 


Version 


5.1 


3.4 


2.0 


2.2 


2.6 


5.5 


Pre-formatted reports 


X 


X 




X 


X 




Query tool for less- 
advanced users 


X 


X 






X 


X 


Stored queries 


X 




X 


X 




X 


Online student work 
samples 












X 


User discussion boards 
or user meetings 




X 




X 




X 


Accepts data formats in 
addition to ASCII 


X 


X 




X 


X 


X 


Variable set customized 
to fit school needs 


X 


X 




X 


X 


X 


Company will house data 


X 


X 


X 


X 


X 


X 


School may house data 


X 






X 


X 


X 


Company helps collect 
data 


X 


X 




X 


X 


X 


Reports SIF compliance 




X 


X 






X 


SIF -certified 














Number of districts in 
use 


35 


20 


15 


92 


28 


110 



Account from SchoolNet 

SchoolNet (www.schoolnet.com) is a company describing itself as providing educational tech- 
nology designed to help schools meet their educational goals. Three modules are available with 
SchoolNet: Account, which pennits analysis of student data; Align, an instructional management 
tool; and Outreach, a school district portal designed to facilitate communication between schools, 
teachers, and the public (e.g., parents, community leaders). To analyze student data, SchoolNet 
offers the Account module, focusing on ease of analysis and presentation. Schools can opt to 
purchase only Account; they need not buy all three of the SchoolNet modules. In September 
2003, we reviewed Account, Version 4.0. 
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Data management. Once a school contracts with SchoolNet, representatives 
meet with school personnel to decide the most practical way to handle the school’s or dis- 
trict’s data. In choosing which variables to include in the SchoolNet database, schools 
may choose from a standard set of variables offered by SchoolNet. SchoolNet personnel 
report that although this variable set meets the needs of most schools, custom develop- 
ment is available in many cases for schools that wish to track data items not included in 
the standard set. Schools then gather and clean their data to be sent to SchoolNet to create 
the school’s database; SchoolNet personnel will work with district contact people on this 
process if needed. SchoolNet accepts many common data formats. Upon receiving the 
data, SchoolNet personnel execute an automated cleaning process and notify the school if 
there are changes that should be made. Schools then continue to maintain their Student 
Information System (SIS) and update their SchoolNet database from the SIS as often as 
desired; in fact, some districts execute nightly automated updates. The school can opt to 
house the database on its own servers or on SchoolNet servers. 

Pre-formatted reports. Account features a set of pre-formatted reports that come 
with the implementation of the software, and SchoolNet representatives work with school 
personnel to decide which of these reports to make available for the school/district. Addi- 
tional reports can be defined by the school and added later, but this must be done through 
SchoolNet personnel. These reports also feature broad drill-down capabilities; the user 
can click on any summary in the report and get information about the individual students 
who make up that particular summary. The drill-down capacity enables users to look at 
any available data on the individual students, including historical data as far back as the 
database pennits. All reports can be printed directly from Account or copied and inserted 
into documents. 

Included in the standard set of reports is a set called the “AYP Stoplight Reports,” 
which provides information on Adequate Yearly Progress (AYP) measures from the 
NCLB legislation. In these reports, groups of students who met state AYP benchmarks 
are color-coded in green (passing). Students not meeting AYP benchmarks are color- 
coded in yellow (near passing) and red (failing), based on a “Student Proficiency Ranking 
Index” developed by SchoolNet personnel. As with other reports, the user can click into 
the AYP reports to get more information on the individual students within the three 
groups. 

Query tool. The user query tool in Account is tenned a “custom report builder,” 
providing the means for users to explore data in an ad-hoc fashion and build reports simi- 
lar in appearance to the pre-formatted reports. The query tool is easy to use and is de- 
signed for users of all levels of technical expertise, employing user-friendly check boxes, 
pull-down menus, and other familiar web-form elements. The tool provides access to the 
entire range of the student database and allows users to browse through student data with 
few limitations. Reports built through the query tool also feature clear presentation and 
drill-down capabilities found in pre-formatted reports. Users can run more complicated 
analyses through a function in which data is exported from Account to Excel, SPSS, or 
another package of the user’s choice. The Align module for instructional management 
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allows more detailed assessment queries than does the Account module, but schools must 
have purchased both modules to enable this sort of access. 

SIF. SchoolNet is a member of SIF. Representatives report that SchoolNet soft- 
ware is SIF-compliant and SIF-ccrtified. 

Other. SchoolNet has available electronic discussion boards for users through the 
Outreach component of their software. These discussion boards are available within par- 
ticular districts, although SchoolNet personnel are developing the capability to provide 
discussions between all SchoolNet users. 



Data Miner from Chancery 

Chancery (www.chancery.com) is a company with a focus on school Student Infonnation 
Systems, offering the Chancery Student Management System (Chancery SMS) for this 
purpose. As a service to schools, Chancery offers a reporting product called Data Miner 
that allows ad-hoc querying of infonnation within Chancery SMS. In October 2003, we 
reviewed Data Miner, Version 2.0, working with Chancery SMS, Version 4.1. 

Data management. Chancery SMS is a Student Information System (SIS), so the 
involvement of Chancery personnel in data preparation and management is necessarily 
very involved. In the initial stages of the relationship, Chancery representatives visit the 
district and conduct a data needs analysis, spending a few days meeting with district deci- 
sion-makers in a data planning process. Once this process is completed, Chancery repre- 
sentatives help install the Chancery software on school computers and help school per- 
sonnel get Chancery SMS up and running. Since Chancery is an SIS, schools must house 
and manage the data on school computers. 

Pre-formatted reports. Chancery offers a set of pre-formatted reports with Chan- 
cery SMS. About 50 reports come with the implementation of the software, and schools 
can request that custom reports be added. Most pre-formatted reports are easily accessible 
through a set of user-friendly li nk s on the “dashboard,” (the user’s initial screen), and 
some are also available through a query screen, which requires some brief specification. 
These reports contain a variety of information at the district, school, and student level and 
are presented in printable and exportable graphs or tables. Chancery personnel report that 
schools can request “drill-down” capacity in some reports, the capacity to click on an 
area of a graph or table and gain more information about the underlying students. 

Query tool. The Data Miner query tool included with Chancery SMS is aimed at 
more advanced users. Chancery personnel point out that since the focus of Chancery 
software is on student data management, there has not been a great focus on development 
of presentation tools. As a service, however, Chancery personnel developed the Data 
Miner to handle queries often perfonned at the district level. This query tool is powerful, 
accessing any data in the SIS, and could be used by lower-level users after some training. 
The final presentation of the data is good, with access to tables and graphs, along with a 
drill-down capacity to enable the user to explore subgroups of students. Reports formed 
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by the query tool can be exported to Excel, but the user cannot print directly from the 
Data Miner. Users can store queries for later use. 

SIF. Chancery is a member of SIF. Chancery personnel report Chancery SMS 
tools are SIF-compliant; they are not yet listed as SIF-certified. 

Other. Through the data available in Chancery SMS, the Data Miner has the ca- 
pability to link individual teacher-collected data, such as grades and class percentages, to 
other student data, such as assessments. 



DataPoint from NSSE 

The National Study of School Evaluation (www.nsse.org) is a non-profit educational re- 
search organization that promotes school improvement planning, currently focusing on 
data based decision-making. To perform data analysis, NSSE 6 offers the DataPoint Web- 
based Software for Knowledge Management and Data-Driven School Improvement 
Planning software package (www.nsse.org/datapoint/index.cfin). DataPoint is integrated 
with the NSSE school improvement plan, and offers an improvement planning compo- 
nent in addition to the school data component. In September 2003, we reviewed Data- 
Point (no version number available). 

Data management. Since the focus of NSSE’s programs is to help schools build 
capacity to use data, the data collection and management process intentionally involves 
more school responsibility than that required by most other companies. In the early stages 
of data gathering, NSSE representatives work with school personnel to identify data 
components for inclusion in DataPoint. Data components include not only the traditional 
student measures, but also measures of organizational effectiveness and measures 
pertaining to teaching, where available. School personnel then gather data into ASCII 
format to be loaded into the school’s DataPoint database. Schools are responsible for 
maintaining the database from this point forward. NSSE representatives report heavy 
involvement with school personnel during the identification and gathering processes and 
initial data management because NSSE personnel believe that working closely with 
school personnel during the initial startup phases provides better preparation for long- 
term, efficient data use. After the initial phases, data input into DataPoint can be done by 
anyone with proper pennissions. For instance, designated district personnel might input 
district data accessed by everyone, a particular school might input data accessed only by 
personnel at that school, and a teacher might have access to input his or her own data to 
be used only by him/herself. User- inputted data is linked to the larger, shared school 
database, although there currently is not full integration between user-inputted data and 
the common school data. 

Pre-formatted reports. Pre-fonnatted reports are not available in the initial im- 
plementation of DataPoint, but are available as created by school personnel. After the 



6 Note that the National Study of School Evaluation (NSSE) is a separate organization from the National 
Society for the Study of Education (also NSSE). Both are not-for-profit organizations. 
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DataPoint product is running, pennitted school personnel can run queries and save these 
queries as reports to be available to all users. 

Query tool. The query tools in DataPoint allow the user to build student data re- 
ports in many graphic or tabular forms, disaggregated by a variety of demographic vari- 
ables or for individual students or groups of students. A strength of the DataPoint soft- 
ware is the guidance offered the user through the querying process; the depth of this 
guidance also creates many steps in creating a data query. An interesting querying option 
is the Student Datacard Report, which presents all available data for an individual stu- 
dent. This report can be written to one document and printed, or viewed through the 
software, grouping common data (e.g., achievement measures, student characteristics) 
into tables. A recent addition to the DataPoint query tools is the Student Data Analyzer, 
which allows for statistical comparison of disaggregated results. 

SIF. NSSE personnel report that the DataPoint software is not SIF-compliant. 

Other. NSSE representatives report that the capacity to scan and store student 
work samples is offered in DataPoint, although this capacity is not commonly used. 
NSSE representatives also report an emphasis on the use of data outside the traditional 
student measures of demographics and assessment results. These data include qualitative 
and quantitative measures of school and teacher effectiveness and student performance, 
often including data gathered through surveys, interviews, and observation. As part of the 
NSSE school improvement plan, NSSE representatives work with school personnel to 
collect and use these data to efficiently inform school practice. 



Ease-e from TetraData 

TetraData Corporation (www.tetradata.com) is a company that specializes in software for 
education, offering the Ease-e Analysis Suite for analyzing student data. In September 
2003, we reviewed Ease-e, Version 4.5. 

Data management. Before contracting with TetraData, schools may opt for a 
pre-contract service, where TetraData personnel consult with school personnel to help 
identify which paths toward data use will best help that school. After contracting with 
TetraData, a “data discovery process” begins, where TetraData personnel work closely 
with school personnel in identifying, specifying, and gathering data. TetraData personnel 
describe involvement in this process as ongoing, since school databases often evolve as 
schools make use of data and uncover new needs. Data are sent in any form to TetraData, 
where data cleaning is done during the building of the school’s database. Schools con- 
tinue to maintain the school Student Information System, updating the TetraData data- 
base as desired. Schools may choose whether to house and manage the data on local serv- 
ers or TetraData servers. 

Pre-formatted reports. The Ease-e Suite has a set of pre-formatted reports avail- 
able. The Ease-e reports are created by TetraData personnel after consultation with 
school personnel, and extra reports can be requested later by school personnel and added 
by TetraData staff. These reports contain a variety of cross-sectional and longitudinal in- 
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formation, typically at the district or school levels. Information contained in the reports is 
clearly presented and easy to understand. These reports can be printed from the software 
or copied and pasted into documents or presentations (e.g., Microsoft Word or Power- 
Point). 

Query tool. The Ease-e suite offers two forms of querying: the Classroom Ana- 
lyzer, which is less flexible but less complicated, and the Data Analyzer, a more powerful 
tool. Both tools are easy to operate by any level of user, featuring an intuitive, web-style 
interface. The Classroom Analyzer is intended to be used for commonly-executed queries 
and thus offers a more limited choice of display options. This tool is designed to guide 
the user through a simple five-step query process, and TetraData personnel report this 
tool to be very popular. The Data Analyzer allows access to the full set of variables in the 
database, along with a flexible set of options for output and analysis. Users can display 
data longitudinally over any years available in the data set and have “drill-down” access, 
or the capacity to click on sets of aggregated information to find out more about the un- 
derlying students. Output from the Data Analyzer is printable and exportable to Excel and 
other software packages, and statistical analyses as advanced as linear regression are 
available. Users can also store queries from the Data Analyzer for later use. 

SIF. TetraData is a member of SIF and reports that the Ease-e components are 
SIF-compliant. Ease-e is listed as SIF-certified in the SIF Certification Registry. 

Other. Ease-e users also have access to an electronic discussion board that en- 
ables them to communicate with other Ease-e users. 



EDsmart 

EDsmart (www.edsmartinc.com) is a company that describes itself as founded on the be- 
lief that knowledge of the relationships between educational outcomes is a key to improv- 
ing student achievement. The primary focus of the company is on data analysis and pro- 
viding schools with ongoing help in formulating analysis questions, creating reports, and 
other tasks related to data inquiry. Accordingly, EDsmart seeks to provide data based 
technology for school improvement through its EDexplore software. EDexplore is pack- 
aged with a School Improvement Reports module and with EDdiscover, a new data ag- 
gregation tool. In September 2003, we reviewed Edexplore, Version 3.2, with Versions 
1.0 of the reports module and of EDdiscover. 

Data management. Although not a data warehousing company, EDsmart is at- 
tentive to data warehousing efficiency, implementing a data model that EDsmart person- 
nel assert is very effective at customizing a data warehouse for each school. EDsmart 
personnel work with schools during the data collection process, beginning with the “data 
inventory,” a structured questionnaire designed to help schools identify every possible 
data source that they may have available. During the data identification and collection 
process, EDsmart personnel help school personnel decide which variables will go into the 
school’s database. When the data are ready, school personnel send the data in any elec- 
tronic format to EDsmart to be entered into the school’s database. EDsmart personnel 
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execute a data cleaning process that is both automated and manual, then notify the school 
of changes necessary in the data. School personnel continue to maintain the school’s Stu- 
dent Infonnation System and send periodic data updates to EDsmart, typically about four 
times per year. In most situations, EDsmart houses and maintains the data warehouse. 

Pre-formatted reports. The report module with EDexplore is very flexible, offer- 
ing pre-formatted reports with the attractive feature of “drilling into” the reports (i.e., in 
some instances, users can click on areas of the report and gain more specific infonnation 
about the selected area). For example, if a report lists students not proficient on a particu- 
lar math assessment, the user can click on a student’s name in the report and gain more 
information about that particular student. In addition, EDexplore comes with reports spe- 
cific to the NCLB requirements. 

Query tool. The EDexplore query tool is intended for advanced users and thus 
requires some training and familiarity in order for users to comfortably use it. This tool is 
flexible and offers many different forms of disaggregation and analysis. Users with 
proper access can store EDexplore queries in the form of a report and make these reports 
available for other groups of users. EDsmart has recently released the EDdiscover query 
tool, aimed at administrators, and offering easier access than with the EDexplore query 
tool. EDsmart personnel report that after a half day of training, users are easily able to 
navigate through EDdiscover. The EDdiscover tool operates using “sheers,” or a list of 
variables that users can drag and drop to build a query. Results produced by the EDdis- 
cover software typically display both tables and graphics, which are clear and easy to un- 
derstand. Users can also drill through these results for further ad-hoc exploration. 

SIF. EDsmart personnel report that EDexplore is SIF compliant and will pursue 
SIF certification through a partnership with eScholar (see below). 

Other. EDsmart offers electronic user discussion groups, along with periodic user 
meetings, enabling users to exchange ideas about data analysis with EDsmart software. 
EDsmart also offers fee-based data mining consulting for schools that wish to use the 
procedure to probe their data further. Additionally, EDsmart features the capability to 
link teacher grade book information to the common core of student data, on a custom ba- 
sis. 

In September 2003, EDsmart announced a joint developing and marketing part- 
nership with eScholar. This partnership offers schools that contract with EDsmart the 
data presentation and school improvement strengths of EDsmart, along with the data 
warehousing strengths offered by eScholar. 



eScholar 

eScholar (www.escholar.com) is a company with the declared focus of bringing data 
warehouse solutions to education. eScholar personnel report that the company does not 
seek to be a comprehensive solution for aggregating, disaggregating, and presenting stu- 
dent data, but seeks to provide schools with a sound data warehousing system that most 
efficiently integrates all of a school’s data. Still, eScholar does provide data analysis and 
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reporting tools with their warehouse. In September 2003, we reviewed eScholar, Version 
5.0, with the eScholar Vista Express and Vista Advanced reporting tools. 

Data management. With a focus on providing the best data warehousing solution 
possible for schools, eScholar is necessarily concerned with providing a very efficient 
“data model” (i.e., the setup for data access), and reports extensive, ongoing research into 
identifying and updating a data model that works best for integrating educational data. 
eScholar personnel initially work with schools through a “readiness workshop” that takes 
place before a school chooses to contract with eScholar. The readiness workshop is de- 
signed to explore all the data needs a school may have and to outline clearly the resources 
a school must provide (e.g., cost, personnel, data). Once a school chooses eScholar, any 
further data identification is done with the school. Data collection begins (schools can 
contract with eScholar to help with data collection), then the school sends their data in 
any electronic fonn to eScholar to create the school’s eScholar data warehouse. Extensive 
cleaning and analysis is done at eScholar and the school is notified of any mistakes or 
changes that might affect this data. Schools can contract with eScholar to house and man- 
age the data warehouse, with the school providing periodic updates; alternately, schools 
can choose to house the data themselves. Because of their commitment to their data 
model, eScholar specifies a set of variables schools can choose to include in their data 
warehouse. Although eScholar personnel report offering nearly any data choice that 
schools ask to include, it is still possible that a school might ask for inclusion of a vari- 
able not on eScholar’ s template. eScholar personnel report a positive response to most 
requests, noting that improving options for one school ultimately improves data options 
for all eScholar customers. 

Pre-formatted reports. To produce pre-formatted reports, eScholar offers the 
newly released Vista Express reporting tool. Reports from Vista Express are not purely 
pre-formatted, as the Vista Express tool requires a small amount of specification by the 
user to define the sample. However, sample filtering is minimal and uses pull-down 
menus that require no training. Reports produced by Vista Express typically contain a 
table and graph, are clear and easy to read, and can easily be printed or saved. In certain 
areas, the reports feature drill-down capabilities to explore students within a specific ag- 
gregation. Content offered in these reports is typically aimed at administrators, but teach- 
ers also will find many Vista Express reports useful. One feature particularly attractive 
for teachers is the “student profile,” which displays student history data such as test 
scores and grades, typically found in the student’s permanent record. 

Query tool. eScholar offers a query tool called Vista Advanced, designed for ad- 
vanced data users. The Vista Advanced query tool is business-like in appearance and will 
provide the flexibility and querying power desired by advanced users, but is not likely to 
be popular with less-advanced users. With a stated focus toward data warehousing effi- 
ciency, eScholar management has chosen to focus resources on data efficiency and on 
partnering with other software companies offering tools designed for presentation. Thus, 
districts or schools that wish to provide data access for less-advanced users can take ad- 
vantage of partnerships held by eScholar with companies such as EDsmart and SchoolNet 
that offer more user-friendly presentation products. 
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SIF. eScholar is a SIF member and representatives report heavy involvement 
with the ongoing development of SIF. EScholar software is SIF-compliant and is listed as 
SIF-certified in the SIF Certification Registry. 

Other. eScholar offers a second warehousing option called Violet, which focuses 
entirely on reporting for NCLB requirements. Schools can contract to run eScholar, a lar- 
ger tool that will include NCLB reporting variables along with all other school data, or 
Violet, a smaller tool that includes only NCLB variables. 

In September 2003, eScholar announced a joint developing and marketing part- 
nership with EDsmart. This partnership merges the data warehousing strengths offered by 
eScholar with the data presentation and school improvement strengths of EDsmart. 



QSP from CRESST 

The National Center for Research on Evaluation, Standards, and Student Testing 
(CRESST, www.cse.ucla.edu) is an educational research organization concentrating on 
assessment issues and their application to educational improvement. Through this mis- 
sion, CRESST produces many research products, one of which is the Quality School 
Portfolio (QSP). CRESST offers both a PC-based and a web-based version of QSP. Inter- 
ested users can download the desktop version of QSP free at qsp.cse.ucla.edu. Access to 
the web version is given through CRESST, and requires participation in a training pro- 
gram, along with optional participation in a research study. The QSP software consists of 
two modules: the Data Manager, with which users can manage and aggregate student 
data, and the Resource Kit, which provides tools for collecting data to assess areas such 
as Curriculum and Instruction and Parent Involvement. In September 2003, we reviewed 
QSP, Web Version 4.3; earlier, we had reviewed the currently available desktop product, 
Version 2.0. The programs carry many similarities, so the following discussion will de- 
scribe features common to both QSP products, noting unique characteristics of either the 
web or desktop version. 

Data management. QSP is available free of charge. However, CRESST does not 
venture to offer any data support in terms of cleaning, housing, or incorporating data. In 
order to employ the desktop version of QSP, users visit the website and download QSP 
with instructions for installation. Web users get similar instructions through CRESST 
representatives. For both versions, users then follow data formatting instructions to read 
student data into QSP and are responsible for data preparation, cleaning, and accuracy. 

Pre-formatted reports. Pre-formatted reports are available in the desktop version 
of QSP, but not the web version. These reports offer various disaggregations describing 
school makeup and school performance measures. They require a small amount of speci- 
fication by the user, to identify the type of requested report and the variables to include in 
the report, but are nonetheless easy to access and produce. Reports are printed in either 
tabular or graphic form and are clear and easy to read. 

Query tool. Users access student data through the Data Manager module. This 
module offers many attractive features, one of which is the wide variety of methods to 
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present data. Users build custom reports by identifying the nature of the report through a 
dialogue box, choosing questions such as “What is the average?” or “How many?” This 
feature is very helpful in guiding users through the data aggregation process. To access 
groups for disaggregation, users first build each category and make it available for disag- 
gregation. For instance, if a user planned to specify test averages for all combinations of 
gender by ethnicity, the user would build a category for African American males, one for 
African American females, one for White males, and so on until all possible combina- 
tions were built. Many different options are available for presenting data graphically, and 
the quality of the graphics in QSP is outstanding. QSP also offers a “report builder” that 
enables presentation of individual student data. User-friendly questions and notations 
provided along the way help the user produce the desired report. Additionally, the web 
version of QSP includes the capacity to create reports that link student data to teacher and 
parent data. 

SIF. The QSP program is not currently SIF-compliant. CRESST is a grant- 
funded organization and CRESST personnel report efforts toward finding the proper 
funding to support development in this area. 

Other. The web version of QSP offers an expanded set of capabilities for teach- 
ers. These include the capacity to link teacher gradebook information with the core set of 
student data, along with access to standards information, where available. There is also 
the capacity to include online samples of student work, if the school possesses the tech- 
nology and resources to facilitate such storage. Schools, districts, or states using QSP 
Web may also participate in CRESST-sponsored research that examines issues surround- 
ing data based decision-making. 



Sagebrush Analytics powered by SwiftKnowledge 

SwiftKnowledge (www.swiftknowledge.com) is a company that specializes in providing 
querying, analysis, and reporting solutions for organizations seeking to improve perform- 
ance through data. Their education product, also called SwiftKnowledge, is designed to 
provide educators with easy and flexible access and presentation of student data. Re- 
cently, SwiftKnowledge partnered with Sagebrush Analytics to provide comprehensive 
student data and professional development software to school districts. Under this part- 
nership, the SwiftKnowledge software was retained as the tool for reporting student data. 
In September 2003, we reviewed SwiftKnowledge, Version 5.1. 

Data management. In the initial phases, SwiftKnowledge personnel work with 
school personnel to help decide which student information to make available in the 
school’s database. There is a core set of student data that most schools employ and 
SwiftKnowledge personnel will also customize the data set to fit school needs. The Sage- 
brush/SwiftKnowledge software accesses the school’s Student Infonnation System (SIS) 
or other electronic data stores through a Microsoft SQL server, so there is no separate da- 
tabase to create. SwiftKnowledge personnel help school personnel with any data cleaning 
needed, then help with installation of the new hardware and software. From this point, 
schools are responsible for data updates and maintenance within the SIS. Data, hardware, 
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and software are typically housed and managed by the school, although SwiftKnowledge 
offers such services when necessary. 

Pre-formatted reports. Users access pre-fonnatted reports through the Swift- 
Knowledge “dashboard,” a home page of sorts. On the dashboard, there are a number of 
different pre-formatted reports, typically about five, which the school has chosen to make 
available for users at a particular level, and which change from time to time. In most 
cases, these reports have been created by school personnel responsible for report genera- 
tion, although this is a service available through SwiftKnowledge. Depending on the na- 
ture of the report, schools may choose to make available a static report, or one that allows 
the user “drill-through” capability to click on areas of a report and see more information 
about the underlying students. All reports are printable and are exportable to other soft- 
ware such as Excel. 

Query tool. Contained in the SwiftKnowledge package are two types of query 
tools. One tool deals with a list of variables that are most commonly used (e.g., gender, 
ethnicity, assessments), while the other tool allows access to the full range of variables in 
the database. Although not necessarily aimed at the casual user, these tools nonetheless 
are accessible to all users and are not difficult to learn. Additionally, there are built-in 
help tools accessible to all users. These tools are flexible and powerful and allow data to 
be presented in either tabular or graphical fonnat, with many user-friendly options avail- 
able to customize data presentation. Data can be displayed and disaggregated longitudi- 
nally, for any number of years. Querying can also be done in “drill-down” fashion, such 
that users click on aggregated information to gain more specific information on students 
making up the aggregation. Users can also save queries for future use, and can print out- 
put or export to Excel or similar software. 

SIF. SwiftKnowledge personnel report that the SwiftKnowledge application is 
not SIF-compliant. However, Sagebrush Analytics, the new partner with SwiftKnowl- 
edge, is a member of SIF. 

Other. An appealing feature of SwiftKnowledge is the user dashboard, aimed at 
providing users with quick, important information at a glance. In addition to reports, the 
dashboard contains an “alerts” section that notes important recent changes in data (e.g., 
one school issues alerts when absences rise above a particular level). In addition to dis- 
playing quick information, the dashboard serves as an initial entry point to data, with ac- 
cess to queries and further report information. The dashboard is also customizable. 



SAMS from Executive Intelligence 

Executive Intelligence (www.execintelligenceinc.com) is a company that offers data 
warehousing and reporting tools for education. Accordingly, this company focuses on 
efficient and comprehensive warehousing of school data, and in presentation of this data 
using the Student Achievement Management System (SAMS). In September 2003, we 
reviewed SAMS, Version 3.4. 



29 




Data management. Representatives from Executive Intelligence meet with school 
decision-makers to decide which data elements will best serve the school’s needs. Once 
the variable set is identified, data are extracted from the school’s Student Information 
System (SIS) into the school’s SAMS data warehouse. Extraction is accomplished with 
the ASIX software from Executive Intelligence. After Executive Intelligence personnel 
install ASIX, schools continue to maintain the SIS. Automated uploads into the data 
warehouse are executed with the ASIX software as often as schools choose (typically 
nightly). An automated process is used to check data for invalid responses and the school 
is notified of data that need attention. 

Pre-formatted reports. Pre-formatted reports in SAMS require a small amount of 
user specification because the reports are populated dynamically with real-time data. 
Specification is easy, involving only the selection of a report topic and some basic defini- 
tion of report parameters, such as school or classroom. Output is clear and easy to under- 
stand, and appears in tabular form, with the option to output in an Excel spreadsheet to 
create charts or do further analyses. Reports can be printed from the screen or exported to 
PDF format. In addition to accessing aggregations at the district, school, or classroom 
levels, users may drill into reports to see more information about the underlying students, 
schools, or classrooms. Users also have access to a help screen that provides definitions 
of elements of reports, such as content standards from assessments. 

Query tool. SAMS users may access individual student information along with 
disaggregated group summaries through a user-friendly query tool. This tool uses check 
boxes and pull-down menus to offer users guidance in formulating queries, and seems 
quickly learnable by any level of user. Information is displayed in tables, and users are 
able to drill down through any data summary to see infonnation about the individual stu- 
dents making up the summary. Data disaggregation can be done with any student variable 
available in the warehouse, but each assessment can be disaggregated by up to two vari- 
ables. At the student level, a “student profile” is available. Akin to an official student re- 
cord, this profile thoroughly summarizes information on the student, such as demo- 
graphic information and historical test scores, and is a helpful tool for teachers wishing to 
gain familiarity with individual student histories. The query tool allows for display of 
longitudinal information, so the user may examine data trends over time at the student or 
aggregate level, and allows for simultaneous display of test scores, so the user may exam- 
ine longitudinal trends and associations between assessments. Data presentation is avail- 
able only in tables, and can be viewed in HTML, PDF, or Excel format. Users desiring 
graphs and other representation may export report data to Excel and create graphs exter- 
nally. 

SIF. Executive Intelligence personnel report SAMS to be SIF-compliant in that 
SIF records can be read and produced by SAMS, but they have not gone through the SIF- 
certification process with SAMS. 

Other. Executive Intelligence personnel report the development of a learning 
management tool, which links student data to assessment standards and a student’s Indi- 
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vidual Learning Plan. This tool will be offered as an aid to teachers wishing to use spe- 
cific elements of assessment information to adapt instruction. 



Scholar Suite from SCHOLARinc 

SCHOLARinc (www.scholarinc.com) is a software development company that focuses 
on helping schools manage and analyze assessment data. SCHOLARinc offers Scholar 
Suite, an analysis tool for student assessment data. In January 2003, we reviewed Scholar 
Suite, Version 2.0; no further updates were available as this report went to press. 

Data management. Once contracting to run Scholar Suite, schools gather their 
data from their Student Information System (SIS) and send it to SCHOLARinc in ASCII 
form. SCHOLARinc then merges SIS data with school assessment data provided either 
by the school or the state. After the initial setup, the data is housed on SCHOLARinc 
servers and schools update the Scholar Suite database as frequently as desired. 

Pre-formatted reports. Pre-formatted reports are not available in Scholar Suite. 

Query tool. The query tool in Scholar Suite is simple in focus, in that it provides 
users with a way to query and create graphical or tabular reports. Queries are available 
for the district, school, or classroom levels. The querying capability is flexible, offering 
any representation of assessment proficiency specified by the school. A variety of disag- 
gregation options are available with the demographic variables in the database, although 
selection of variables for disaggregation is somewhat cumbersome. The graphics are clear 
and offer a wide range of graphical presentation, along with columnar tables of data. 
Scholar Suite has a degree of drill-down capacity from the graphs; at the school and 
classroom levels, users can click on portions of the graphs and obtain the names of stu- 
dents and scores that comprised that particular portion of the graph. Although users with 
proper permissions are able to switch between district, school, and classroom reports, the 
drill-down feature is only available to display student information. Users cannot, for in- 
stance, drill through from district to school to teacher to student level. Scholar Suite also 
offers the capacity for school personnel to enter data and link these to student data, so 
that a teacher, for instance, could enter data such as tests and quizzes, and have these as- 
sessments available for analysis. 

SIF. SCHOLARinc reports that Scholar Suite is SIF-compliant, although 
SCHOLARinc is not listed among SIF-certified applications. 



Socrates Data System from the Center for Resource Management 

The Center for Resource Management (CRM, www.crminc.com) provides services for 
education such as research, program evaluation, standards implementation, and data use. 
One such service provides help in building school capacity to use data for improvement 
and accountability. This service includes the Socrates Data System, comprised of the 
Socrates relational database software and the Socrates Data Web, a customized web por- 
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tal for accessing reports and other infonnation (www.cnninc.com/Socratesl.htm). We 
reviewed the Socrates Data Web, Version 2.2, in September 2003. 

Data management. In the initial phase of the relationship, CRM personnel work 
with school and district personnel to identify the data that are available to be imported 
into the Socrates database. School personnel then organize and send the data in electronic 
form (Socrates accepts common data formats) to CRM to be loaded into the Socrates da- 
tabase. While loading the school data into the Socrates database, CRM personnel perform 
a cleaning process, producing reports of specific data elements and records that need at- 
tention. Once the database is ready for use, CRM houses and manages the data for the 
first year, receiving data updates from the school at time intervals chosen by the school. 
When school personnel feel ready, they then can opt to buy a Socrates site license and 
take over data management themselves. Otherwise, they can contract with CRM to con- 
tinue housing the data. Schools continue to maintain their Student Information System 
and periodically update the Socrates database. These updates can be done as often as the 
school chooses, regardless of where the database is housed and managed. 

Pre-formatted reports. Users of the Socrates Data Web component of the system 
access a voluminous amount of prepared reports available at the district, school, and pro- 
gram levels. If a school requests, classroom-level reports and individual student profiles 
may also be made available. These reports require no specification by the user; the user 
simply selects the report from a list, then either views or prints the report, so the software 
provides a great amount of prepared infonnation with a few clicks. A large number of 
core reports (e.g., disaggregated assessment results, discipline, and attendance) are in- 
cluded, and schools may also request any number of additional reports showing relation- 
ships among items in the district’s database. Reports are arranged by level, such as dis- 
trict or school level, then within each level, specific reports are intuitively arranged into 
groups such as enrollment and absence data, achievement tests, and class grades. Reports 
can be specified in any presentation format a school chooses (e.g., tables, charts, any 
graphical form), and are clear, concise, and easy to read in all of the formats presented. 
Tables are consistently fonnatted from report to report, so the user quickly becomes fa- 
miliar with the appearance of the tables. 

Query tool. CRM personnel report research and experience that indicate that hav- 
ing to learn a query process to access data can be a significant obstacle to data use in any 
data warehouse system. Consequently, the Socrates Data Web was created to provide ac- 
cess to reports (see “Pre-formatted reports”) that CRM personnel believe address the ma- 
jority of data questions. CRM provides training for designated school and district person- 
nel and other users who wish to perfonn ad-hoc queries in the database. 

SIF. CRM personnel report that Socrates does not yet meet specifications for SIF 
compliance. 

Other. The Socrates Data Web also comes with a Resources section that contains 
articles, worksheets and guidelines for using data, and websites on topics of interest to 
school personnel. These works are drawn from educational literature and CRM’s own 
research. CRM personnel report communication tools in the works, such as electronic 
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discussion boards to enable Socrates users to contact each other regarding Socrates and 
other data issues. 



STARS from SchoolCity 

SchoolCity (www.schoolcity.com) is an education software company providing data 
software and e-learning services to school districts. For student data analysis, SchoolCity 
offers the Standardized Test Analysis and Research System (STARS, 
www.schoolcity.com/stars.htm), a tool formerly owned by the American Productivity and 
Quality Center. The focus of the STARS software is on providing disaggregated student 
assessment data. In September 2003, we reviewed STARS, Version 2.6. 

Data management. To begin the data process, district personnel collect a defined 
set of student data comprised of 23 fields of demographic and academic information such 
as gender, ethnicity, and course grades. These data are collected by the school from the 
school’s Student Infonnation System (SIS) and submitted to SchoolCity in ASCII format. 
SchoolCity personnel then obtain student assessment data from the appropriate source 
(typically a state, publisher, or school) and merge assessment data with the school- 
submitted SIS data to create the school’s STARS database. Since the set of 23 SIS vari- 
ables is pre-set, SchoolCity personnel typically do not help schools with data collection 
and identification, although this service is available if needed. Schools also can include 
student data in addition to these 23 fields, but these variables must be submitted sepa- 
rately and imported with the assessment data. In the typical arrangement with SchoolCity, 
schools house and manage their databases, and updates are done every tenn. Services are 
available, however, for SchoolCity to host and manage the data; more frequent update 
options are also available. 

Pre-formatted reports. When users log in to STARS, a set of pre-formatted re- 
ports is available, offering infonnation according to the user level. For instance, a teacher 
may choose from reports pertaining to her/his students, while a principal may access re- 
ports pertaining to the entire school. Examples of report content available in the STARS 
implementation include minimum standards proficiency and average achievement test 
scores, disaggregated by a number of demographic variables. These reports are easy to 
access and understand, and are available in tabular and graphic forms. Reports are print- 
able and are exportable to Excel. 

Query tool. The STARS software offers various methods to query the data. One 
form of querying allows users to run the same reports available in pre-formatted form, 
but selecting certain characteristics of the available sample of students. Users can drill 
through these reports to obtain infonnation on the participants making up certain areas of 
the reports. There is also a more powerful query tool that allows the user freedom to ma- 
nipulate both the sample characteristics and presentation of assessment data. This tool 
provides a variety of methods to present and disaggregate assessment data, with a limited 
range of longitudinal data and multiple assessments. To bolster capacity to view con- 
comitant data, SchoolCity will soon release a new query tool with STARS that allows up 
to six assessments in one report. The six assessments available can be separate tests, tests 
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given over a number of years, or a combination of both. All of the STARS query methods 
are intuitive to operate, using familiar check boxes and pulldown menus. 

SIF. SchoolCity personnel report that STARS is not currently SIF-compliant, 
citing a lack of demand from clients for SIF-compliant software. SchoolCity personnel 
report familiarity with SIF requirements and may move toward SIF-compliance at a fu- 
ture date. 



Virtual EDucation from EDmin 

EDmin (www.edmin.com) is a company with the stated goal of improving schools by 
providing technology that electronically connects all stakeholders surrounding a school, 
including educators, students, parents, and other community members. EDmin offers the 
Virtual EDucation software package (www.edmin.com/products/ved/index.cfm) as part 
of this mission. EDmin describes Virtual EDucation as a “learning management system” 
that enables educational improvement by offering connections between many different 
aspects of school life. Examples of modules contained in Virtual EDucation include in- 
structional management (e.g., lesson planning, grades), school newsletters, access to 
learning standards, online student portfolios, and student data analysis/reports. The stu- 
dent data modules are targeted specifically at presentation and disaggregation of student 
assessment data. In September 2003, we reviewed the student data modules of Virtual 
Education, Version 5.5. 

Data management. Before data collection starts, EDmin personnel meet with 
school personnel to help identify which variables will be imported into the school’s data- 
base. These variables typically consist of any assessment variables a school has, along 
with a set of up to 15 demographic variables for disaggregation purposes (e.g., gender, 
ethnicity). Once variables and their sources are identified, schools collect, clean, and send 
the data to EDmin in any electronic format (tab-delimited is preferred). EDmin personnel 
subsequently perfonn a cleaning process on the data and notify schools of needed 
changes. Once clean, the data are loaded into the school’s Virtual EDucation database. 
The school or district continues to maintain their Student Information System, updating 
the database as often as needed. EDmin typically houses the school’s Virtual EDucation 
database, though the school can choose to house it on their own server instead. 

Pre-formatted reports. The Virtual EDucation package does not contain a set of 
pre-fonnatted reports. 

Query tool. The query tools within Virtual EDucation provide a number of ways 
to build reports on student assessment data. The tool is easy to leam, using pull-down 
menus to define the sample and assessments for the desired report. The tool is a nice 
combination of flexibility and guidance, in that users are allowed to select and group stu- 
dents whom they are pennitted to view, but can click on links to obtain certain types of 
reports about these students. For instance, there is an “at-risk report” available where 
relevant information about students scoring below a specified test score is presented and 
a “progress chart” that displays test score changes for schools, classrooms, or students 



34 




over a given period of time. There is also the capacity to display trends for multiple as- 
sessments simultaneously. A great deal of flexibility is provided by the “drill-down” ca- 
pacity in these reports, enabling the user to click on any aggregated set of data and obtain 
information on the participants who make up the specific group. Most reports are avail- 
able in tabular form, with the option to re-sort tables in many different ways. Graphs and 
charts are typically not available immediately; users who wish to create graphs must ex- 
port the data to Excel or another graph-creating tool. There are also ample links to and 
integration with other modules included in Virtual EDucation; for instance, it is easy to 
link to assessment standards from many result tables. 

SIF. EDmin representatives report that they have not yet begun the SIF- 
compliance or certification process, but that Virtual EDucation is SIF-compliant. Com- 
pany representatives report a desire to wait until more definitions are set by SIF before 
starting the compliance or certification process. 

Locally Developed Software 

School districts sometimes choose to build their own student data analysis system. We 
spoke with three large districts in Broward County, Florida, Cleveland, Ohio, and Hous- 
ton, Texas, that have done so and are currently implementing these systems in their 
schools. We were not able to review these programs, but spoke with representatives and 
viewed presentations of the software. 

These locally built systems appear to offer similar capacities as the publicly- 
available programs reviewed above. All were built by Infonnation Technology (IT) staff 
employed by the district, and were integrated directly into the existing Student Informa- 
tion System. This was supported in different ways. Broward County, for instance, ob- 
tained a grant to partner with IBM for help in building the data warehouse. In Cleveland, 
the project started on a small scale, with IT employees building a limited access tool for 
the database. The Cleveland project continues to grow in scale. All software was devel- 
oped with local needs in mind, and all continue to evolve within this context. 

Houston offers a comprehensive package that enables queries and drill-down ca- 
pacities. The access to a wide range of student data is impressive and the system looks as 
if it is easy to use. Broward County also offers a wide range of access, and has placed 
emphasis on report availability. In addition, Broward County continues to push forward 
user queries, recently rolling out versions of query tools aimed at every level of expertise. 
Cleveland’s program is in an earlier developmental stage than the other two, and offers 
presentation of snapshots of student data with analysis available from a data download to 
a program such as Excel. The IT staff in Cleveland report a heavy emphasis on training 
users on the use of the software and in Excel. 

All three of these local efforts are only available locally, so school personnel must 
be on school grounds to access them. Broward County and Houston report efforts to en- 
able Internet access to their programs, with the necessary privacy concerns kept firmly in 
mind. Cleveland is not moving as quickly toward Internet access. Since users download 
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student data for analysis, such analysis takes place at the PC level, so the data have a de- 
gree of portability. 

All three locations further report a very positive response to these initiatives. All 
report rapidly increasing numbers of new users. In addition, all three locations report 
positive response to training that enables the user to run ad-hoc analyses. 



Discussion 

We have examined issues surrounding a proposition that is prevalent in both the business 
management and school effects literature — that creating a more nearly data- and informa- 
tion-rich environment can help improve practice and performance. School research and 
our own observations indicate that being “data-driven” is a phenomenon more often ob- 
served in the breach than in the basic structure of the typical school. We have also ob- 
served that the absence of data-informed decision-making is not inherently due to educa- 
tors’ aversion to being informed. Rather, the wealth of data potentially available in 
schools is typically not stored in ways that are practically accessible to teachers and prin- 
cipals. 

This wealth of data need not remain inaccessible. There are many software tools, 
both locally built and commercially available, to facilitate access to interpretation of data. 
Advancements in computer technology have created powerful, affordable hardware to 
support these tools, and this technology is getting more powerful and less expensive. 
Software development in this area is producing even more efficient and usable tools for 
educators. 

Many issues remain surrounding the use of student data for decision-making. 
These issues provide ample opportunity for further research within this area, and will be 
addressed in the remainder of this section. In discussing each issue, we outline a series of 
1 1 important research questions, each of which could be addressed through a relatively 
straightforward experimental study. 

Educator Use of Student Data Software 

Throughout this report we have discussed the need for developers to listen to and observe 
the needs of educators. Based on our experience with testing software, interviewing de- 
velopers, and working with educators, we have speculated as to what an excellent soft- 
ware package might provide. A necessary next step is to provide rigorous studies of edu- 
cators’ use of data analysis and presentation software. 

There are several references available that discuss others’ observations regarding 
the nature of the school data process and educator involvement (e.g., Lachat, 2002; 
Feldman & Tung, 2001). However, there do not yet exist controlled studies that examine 
educator use of student data software, and the effect of such use on student outcomes. 



36 




Research Question 1: What practical use do teachers and principals make of 
rich classroom- and school-level data information tools, when they are available? 

Research Question 2: What types and levels of professional development are 
most effective in assisting teachers and principals in becoming facile users of such data 
and data tools? 

Research Question 3: As educators become skilled users of rich, multi- 
dimensional student data at the student, classroom, and school levels, how do these skills 
affect their teaching? 

Research Question 4: To what extent do these changes in teacher and principal 
knowledge and behavior affect student achievement and other desired outcomes? (e.g., 
Can the promise of “ data - and information-richness” for improving students’ academic 
achievement gains be documented in controlled experiments?) 

Building and Maintaining a Data Climate 

Data access is a necessary condition for promoting data-infonned decisions, but it is not a 
sufficient condition. To make the best use of such access, schools must create a climate 
that encourages data use, lends proper technology for efficient access and interpretation 
of data, and provides professional development opportunities to ensure that teachers and 
other educators are able to make the most of their data. 

Without a data-driven climate, even the best access to data is doomed to use by 
only a few unusually interested parties. School leadership must support and expect data- 
driven decision-making. This almost certainly means providing time for such endeavors, 
and promoting opportunities for collaboration and support from other educators. Data use 
should be promoted near the top of an educator’s priority list, to be envisaged as part of a 
continuous cycle of data analysis, lesson planning, lesson presentation, and data gather- 
ing. Further, a data-driven climate must be structured such that data use is an expected 
part of an educator’s nonnal day, as are lesson planning or grading. 

Research Question 5: What administration behaviors, at the district and school 
levels, will be necessary to create and to sustain a data-focused, data-driven decision- 
making climate at the classroom and school levels? 

Research Question 6: What types and levels of professional development are 
most effective in supporting and sustaining a data-focused, data-driven decision-making 
climate at the classroom and school levels? 

Teachers as Researchers 

Providing access to student data to inform educational practice would directly support 
those furthering the ideal of teachers as researchers. Throughout this report we have im- 
plied that it is not necessary that teachers first become experts in statistical data analysis. 
We do not envision most classroom teachers using newfound access to student data to 
employ a full range of inferential statistics, sampling methods, and all the other hallmarks 
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of traditional quantitative research methodology intended to allow inference from a sam- 
ple to a larger population. Quite understandably, most teachers are less interested in sta- 
tistical inferences to broad populations, and most interested in one specific population of 
students — their own. 

Research Question 7: Given broad access to student data histories and addi- 
tional knowledge about their students, what forms will teacher research assume? Will 
teacher examination of student data be descriptive in nature or take on more complicated 
forms? 

GIGO 

In this report, we advocate very broad access for teachers and other education profes- 
sionals, but access of this kind brings its own set of problems that should be addressed. 
Any profession attempting to expand its knowledge base confronts the phenomenon of 
GIGO — garbage in means garbage out. We anticipate that professional development will 
be required to minimize misuse of some unjustified but readily available comparisons, 
and that software should offer restrictions to help enable sensible comparisons. An exam- 
ple of misuse might be the literal comparison of scores on different measures. 

Research Question 8: What forms of professional development and software de- 
signs will most nearly ensure the educationally defensible use of the wealth of readily 
available comparisons offered by broad access to data? 

Beyond Accountability 

Earl and Katz (2002) state that data are best used for improvement purposes, rather than 
for accountability purposes. We believe data use such as that promoted in this report will 
provide a positive complement to current “macro” data collection. Many educators are 
suspicious of data use, often feeling “used” by it (Earl & Katz, 2000). Data use such as 
that promoted here provides a measure of control over the use of this data — a way to use 
data instead of being used by data. Moving beyond accountability promotes intrinsically 
motivated exploration and holds great potential for infonning teacher classroom practice. 

Research Question 9: How will use of student data beyond accountability affect 
teacher self-efficacy and motivation toward further data use? 

Teacher Preparation 

Most teacher preparation programs currently do not provide preparation in the area of 
data based decision-making. Teacher preparation researchers for years have stressed 
teacher inquiry into practice and effects (Goodlad, 1990; Holmes Group, 1990), and the 
marriage between data use and teacher preparation is a natural one. The current genera- 
tion of college students is the first to have grown up with computers, so experience with 
computers and instant access to information is generally not the obstacle for many pre- 
service teachers that it might be for some senior educators. 
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Teacher preparation programs are beginning to implement courses aimed at data 
use through action research and assessment methods. These programs would additionally 
benefit from recognizing the utility of software such as that described here and quickly 
expanding preparation of teachers to engage in data based decision-making. 

Research Question 10: When teacher candidates are prepared to use student 
data, will they continue data practices once they enter the teaching profession ? Will these 
teacher candidates become more effective teachers than candidates not prepared for data 
use? 

Online Portfolios 

Currently, the majority of both locally and commercially developed data warehouses fo- 
cus on quantified data. Only a few allow for the storage of more “authentic” measures of 
students’ academic performance. The historic argument against storing full student port- 
folios was one of storage costs. However, the cost of data storage has plummeted and 
continues doing so, making the cost argument less compelling. 

Research Question 11: Will the presence of more nearly “authentic” student 
work, such as handwriting, artwork, essays, or PowerPoint presentations for classes 
prove sufficiently valuable to teachers and administrators, where such storage is avail- 
able, to create a demand for such functionality? Will the additional functionality, where 
present, serve to further data use and student achievement? 



Summary 

We believe the development of tools for classroom- and school-level use of student data 
represents an important opportunity for informing educational practice at both the school 
and classroom levels. The opportunity for today’s educators to tap into the great wealth 
of presently accessible student data no longer needs to be an abstraction. 

Although the use of this technology is not yet widespread, efficient, practical im- 
plementation of data management technology is available today. The use of data to man- 
age information has been widespread in other fields for a number of years. Both local dis- 
tricts and for-profit corporations have spent much of the last decade developing and refin- 
ing products that facilitate the storage, analysis, and presentation of educational data in a 
range of levels. 

There are many questions yet to be answered as to how the evolution of these 
technologies will proceed, how they will and should be managed, and what standards will 
emerge. In this report we have proposed a series of areas both for additional practical use 
of these technologies and for research on how to maximize that productive usage. 
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Appendix A 



There are also a small number of software companies that we were unable to review for this re- 
port, but who provide analysis of student data. Achieve, from Project Achieve, offers analysis of 
student data related to state learning standards, and EADMS, from Adrylan Communications, 
offers analysis of student data by way of pre-formatted reports. Unfortunately, we were unable to 
get a hands-on demonstration of these two products and the opportunity to actually operate the 
software was a criterion for inclusion in this study. Pearson Education Technologies (formerly 
NCS Learn), the company that offers SASIxp, lists on their website software that analyzes stu- 
dent data, but repeated attempts to make contact with various company personnel for a demon- 
stration were unsuccessful. In addition, we reviewed a demonstration of the Achievement Man- 
agement System from TumLeaf, but TumLeaf requested not to be included in these reviews. It is 
our hope that subsequent updates of our work will include these vendors. 
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Appendix B 



To give the reader a better sense of the look and feel of these programs, we asked each vendor 
included in our reviews to send an example of their software. This appendix contains “screen 
shots” from vendors who provided examples of their software. 
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Account from SchoolNet 

Below is an example of the AYP Stoplight Report available in SchoolNet for NCLB reporting. 
Groups are coded in green if meeting a particular AYP goal, yellow if nearly meeting the goal, 
and red if substantially behind the goal. 



SchoolNet Adequate Yearly Progress Stoplight Report 



School Year: 2002-2003 
All Grade Levels 

AYP Goal: 2006-2007: ELA: 50 %, Math: 50 % 



*jsi 



powsrod by 

schoolnet 



NCLB Category 



Nath 



Summary 



Female 



Male 



Race: Asian Pacific 



Race: Black 



Race: White 



Ethnicity: Hispanic/Latino 
Ethnicity: Other 
Special Education 



LEP 



Socio-Economically Challenged 



Migrant 



O Insufficient data Q AYP Goals Met O Area of Concern 0 AYP Goals Not Met 
Numbers represent number of students needed to meet AYP Goal 
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DataPoint from NSSE 



Below is the Main Menu from DataPoint. Users may choose a variety of options, including cus- 
tom analyses, searching for individual students, and resources from the NSSE school improve- 
ment plan. 




46 






Ease-e by Tetra Data 

Below is output from the Ease-e Data Analyzer, showing an example where counts of students 
are disaggregated by free lunch program, ethnicity, and gender. Output is initially given in tabu- 
lar form, with other forms also available (graph shown on inset). 




AMS 

• _) Lone act Seer* 

- Perlman:* L*s«r 



• rulNof Al'rmpl 

• Psformance Le<« U. 

. 9 W*ng 

• QScJ*d Score 

• O Total Sea# 

. i_J V&UnQ Tf* 

Attendance 

- il*i 

- Court* 

• rnurw firwfcn 

■ DAP 

Dwupane 

0 *6 Wematw* mnucw 

0 Cut-cf-Sctiea SUipOTCCnS 

■ hpa 

• Mob*** 

- GAI9 

- saiv cluster* 

- Srltool 

- 'ilmbiit 

0 Attendance Cod* 

• 0E«noui 

9 MoguJ entry oat* 

9 Etngul E’i: Cut* 



• 0 Cngfch Protoercv 

• 0EthncA> 

• 9K5«rwrnt 


B 




. 0 r.-twl 




• 0 Glide 




• f» Home language 




• 9 Mcmtteo 





_/© @© XlMlS¥ * 




200C-2001 
Value XOoarh 



20C 1-2002 
vdue %'jowtri 

Ti 4706% 



Urcn CHnoam 
P« 

Tree 

Pt*0U*d 

Uren Status IHncwm 



l**cn Stat* irinoam 



0 Homeroom 

9 1 in ■/<*■• rv» 

0 tnstiunond Oettixj 
9 l*st Sc Tool Attondod 
9 uranic A.nmutrm'. 
9 Lurch Status 
0t*?e-it Statu* 

9 tiatrre Anencan 
0 Orray lanx-eoo 







American Indian. Female 




American Indian Vale 




Alien, 'emela 




A»an. Stale 




ateck.Female 




Black. Male 




Hr* pur ii .FrrnaW 




Hispar ic .Mat 




Wh4t>/iriii«li? 




Whfejlteb 




Other Ell <n city .Ferial* 




Ethnic ty Ur known .Female 




Ethnic tv Urknowr.r/al* 



Al TW>e.L nth -I ■Tut lamcww 
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EDsmart 



This screen shot illustrates the EDdiscover module from EDsmart. Analysis variables are chosen 
from “sheers” on the left, and aggregations are presented in tables and graphs on the right. 
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eScholar 



Below is an example of output from the Vista Express reporting tool. This particular report pre- 
sents assessment achievement disaggregated by attendance. 
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QSP from CRESST 

This example illustrates the capacity of QSP to aid in producing a report of varied academic in- 
formation on individual students. 



qsp: 



George Washington High School 

David Jones /Semi, 01-02 



help 

logout 



Home 


— 

Gradebook 


Students 


f _ _ ) 

Reports 


— 

Groups 


Goals 


F 


Class: |Per2 English (Semi, 01-02) 


Group: 


All School 









Name |l - 12 T] 

Jones, Seth 

Jones, Stuart 

Jones, Guadalupe 
Jones, Stan 
Jones, Owen 

Dunks, Ashley 

Jones, Aba 
Jones, Simon 
Jones, Fred 
Jones, Aaron 

Jones, Jim 

Jones, Sue 

29 active students 



Students 



History 



Portfolio 



Parents 



Item 1 1 - 10 


Subject A 


Teacher 


Date 


Grade 


Concept Map 


History 


Cummings 


08/16/99 


A 


Project 1 


History 


Cummings 


09/11/99 


B+ 


Eovot Pa 




History 


Cummings 


04/04/00 


A 


Pottery 1 


student work 

assessment 

exemplars 


History 


Cummings 


05/14/00 


A- 


Rooseve 


History 


Bishop 


10/02/00 


B+ 


Math Prc 


Math 


Fujizawa 


09/05/99 


A 


Presenta 


tion via... 


Math 


Fujizawa 


09/10/99 


A 


Essay 1 


Math 


Washington 


11/24/00 


A- 


Concept Map 1 


Social Studies 


Sharad 


09/28/99 


B- 


Concept Map 2 


Social Studies 


Stevens 


04/16/01 


B+ 


17 items 
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SAMS from Executive Intelligence 

Below is an example from SAMS, summarizing test performance for an individual student. A 
summary is given for standards within the assessment, along with action information regarding a 
specific standard. 




1ST )one Internet 
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Sagebrush Analytics, powered by SwiftKnowledge 

This is the initial user screen from SwiftKnowledge. Users may access reports on the top, queries 
on the bottom right, and are alerted to recent important information through SwiftAlerts on the 
bottom left. 
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Scholar Suite from SCHOLARinc 

This is an example of simultaneous assessment presentation in Scholar Suite. This particular re- 
port presents proficiency on four different tests, side by side. 
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STARS from SchoolCity 

This shows an example of drill-down from the classroom to the student level. The inset presents 
descriptions for specific assessment objectives. 
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Virtual EDucation from EDmin 

The report shown here enables users with proper permissions to see all of the student achieve- 
ment and performance gains for an entire year, helping to track AYP. Users may click on links 
under “Cluster” to drill down as far as the individual student level. 



Uirtual EDucation 



A Home © Help [2 Sign Off §3 Setup 



O Instructional Management 
O Classroom Management 
O Performance Center 



User: District Administrator 



O Standards 
O Online Portfolio 
O Reports 



O Communication 
O Newsletter 
C Calendar 



O Staff Development 
O Meeting Manager 
O Utilities 



Access Level: District Administrator 



Performance Center 



State/District Assessments 



Select Filters 
Baseline Report 
Longitudinal Report 
At Risk Report 
Group Profile Report 
Progress Chart 
Enter Scores 
Settings 



Multiple Measures 



Select Filters 
Baseline Report 
Progress Chart 
Enter Scores 
Settings 



Performance Profile 



Select Filters 
By Student 
Over Time 
By Standard 



State/District Assessments - Progress Chart 



Cluster: All 

Department: Unassigned 

Teacher: All 
Student: All 



Gender: All 
Home Language: All 
Special Ed Ser Del: All 
Ethnicity Breakdown: All 
Assessment #1 
Battery: CA - STAR 2002 
Assessment #2 
Battery: CA - STAR 2002 



Select Filters 



School: Global Elementary 
Grade: 4 
Class: All 



Ethnicity: All 
Title 1: All 
Spec Ed ID: All 



Filters 

School Nobility: All 
Gate: All 

New Arrivals: All 



District Nobility: All 
Parent Ed Level: All 
Lang Fluency: All 



Level: Task 3 Strand: Read Comp Score Type: Scale Score 



Level: Task 2 



Cluster 


Assessment # 1 


Assessment #2 


Delta 


Default Cluster for District 1 


635 


625 


-10 


North 


653 


576 


-77 


East 


670 


670 


0 


West 


635 


650 


J +15 



' N / A : Not Available 
*DNA: Did Not Attempt 

Go to top of page 



Se lect F ilters 



25% or Lower Between Avg , . . Between Avg 25% or Greater 
than Avg and Lower 25% - 9 and Top 25% than Avg 

P Y B G 



Print Preview 



EDmin.com 



Instructional Management 
Classroom Management 
Performance Center 



Standards District Communication Report Card 

Online Portfolio Staff Development Tracker Utilities 

Reports Curriculum Management IEP Manager 
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